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ABSTRACT 

Aspects of the perception that assessment is a 
large-scale testing program conducted at institutional or state 
levels co determine what students have learned from college is 
questioned. Small-scale assessments conducted continuously in college 
classrooms by discipline-based teachers to determine what students 
are learning in what class are also important. The clrect involvement 
of classroom teachers in the assessment movement is important 
because: teachers need continuous feedback on learning in the 
classroom so they may improve teaching effectiveness; continuous 
feedback is necessary for improvement in teaching and learning; and 
it is important to know more about how students learn. The two broad 
categories of feedback provided b> students to teachers are: (1) 
student evaluations of teaching in which they report their 
observations and reactions to the course and instruction and (2) 
measures of student learning such as tests and daily assignments. 
Sources of bias in student ratings of instruction include student, 
faculty, and course characteristics. Other topics of discussion are 
consistency and agreement on teaching effectiveness, validity, and 
the effect of student ratings on instruction. The effect of feedback 
to teachers on student achievement in the classroom is noted. 
Conclusions and recommendations include: students provide relatively 
unbiased, reliable, and valid information; feedback is much more 
effective in producing change when augmented with consultation; and 
research is needed on the efficacy of providing teachers with better 
methods of classroom learning. The topic of colleagues as sources of 
feedback is considered as follows: feedback from colleagues as peers; 
feedback from consultants and experts; and self-assessment. Contains 
69 references. (SM) 
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Most people think of assessment as a large-scale testing pro- 
gram conducted at institutional or state levels to determine what 
students have learne in college. The purpose of this paper is 
to question almost every aspect of that perception. I believe 
that ve should be giving more attention to small-scale assess- 
ments conducted continuously in college classrooms by discipline- 
based teachers to determine what students are learning in that 
class. 

The advantage of thinking small in assessment is that the 
classroom is the scene of the action in education. If the ulti- 
mate purpose of assessment is to improve teaching and learning, 
then the results of a successful assessment must eventually bear 
directly on the actions of teachers in their classrooms. This 
means that the feedback from any assessment must reach classroom 
teachers and be perceived by them as relevant to the way they do 
their jobs. One way to do that, albeit not the only way, is to 
start in the classroom collecting assessment data that teachers 
consider relevant. 

This paper then is about the use of information that is — or 
could be — collected in college classrooms with the intention of 
improving teaching and learning. We know quite a bit from sev- 
eral decades of research about the uses of feedback to improve 
teaching and learning, and I should like to review that research 
in the context of the 1980 9 s assessment movement. 

Feedback plays different roles in two current modes of 
assessment that are frequently contrasted as assessment-for- 
accountability and assessment-f or- improvement. Assessment- for- 
accountability is illustrated in the remarks of Missouri Governor 
John Ashcroft who chaired the Governor 9 s Task Force on College 
Quality. He said, "The public has the right to know what it is 
getting for its expenditure of tax resources; the public has a 
right to know and understand the quality of undergraduate educa- 
tion that young people receive from publicly funded colleges and 
universities 19 (National Governors 9 Association, 1986, p. 154). 

The assessment-for-improvement perspective is expressed by 
William Turnbull, late president of Educational Testing Service. 
He wrote, "The overriding purpose of gathering data is to provide 
a basis for improving instruction, rather than keeping score or 
allocating blame 11 (1985, p. 25) • 

While the ultimate goal of both types of assessment is the 
improvement of education, feedback is handled quite differently 
in the two models. In the accountability model, feedback is usu- 
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ally public, normative, comparative, and competitive. As Gover- 
nor Ashcroft noted, the public hat a right to know, and that 
implies that they have a right to compare programs competing for 
funds. Assessment- for-accountability is also "summative" in the 
sense that it provides a "stop-motion" summary of the situation 
at any given point in time. The purpose of the information is to 
demonstrate the effectiveness of the education and ultimately to 
correct deficiencies. 

In contrast, the role of feedback in the assessment- for- 
improvement model is to provide a continuous flow of information 
that is useful in shaping the process of teaching and learning 
while it is in process. This is generally referred to as "forma- 
tive evaluation," and feedback is most effective if it is not 
made public and emphasizes competencies instead of comparisons. 

A related contrast that I should like to draw in setting the 
stage for a discussion about using feedback more effectively in 
assessment is a distinction between what I shall call "direct" 
and "indirect" models of assessment. Those who, by virtue of 
their positions, are interested in accountability usually have 
indirect responsibilities for teaching and learning. That is, 
they are not on the campus or in the classroom, and must try in 
whatever ways they can to influence the behavior of those who are 
in a position to affect teaching and learning. It follows that 
the further removed one is from the scene of the action in teach- 
ing and learning, the more one is dependent on manipulating 
reward and punishment to bring about desired ends, states manip- 
ulate rewards for institutions; presidents manipulate rewards for 
departments, and department chairs manipulate rewards for teach- 
ers. 

Without arguing the effectiveness of the indirect model for 
educational improvement, I want to contrast it with an assessment 
model that emphasizes direct involvement in the educational pro- 
cess, classroom teachers are directly involved in instruction. 
Through their own actions, they can change the quality of teach- 
ing and learning in the classroom. For that reason alone, it is 
important to get teachers as individual* involved in this assess- 
ment movement. 

Although some faculty members are currently involved in 
designing or approving assessment programs, they act as represen- 
tatives of the collective faculty, which is a very different role 
from assessing their own effectiveness as teachers. It is 
assumed in most current forms of assessment that if college 
teachers are made aware that students lack knowledge in subject 
matter areas considered important, the collective faculty will 
take steps to correct the deficiency — usually through making 
changes in the curriculum or requirements. Virtually ignored is 
the premise that much of what is taught is not learned. 

If we could increase the efficiency of education by reducing 
the gap between what is taught and what is learned, it would be a 
remarkable and long- lasting achievement. We need the direct 
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involvement of classroom teachers in the assessment movement for 
the following reasons: 

1. Teachers need continuous feedback on learning in the 
classroom so that they may evaluate, experiment, and improve the 
effectiveness of their teaching and its impact on students 1 
learning. 

2. Continuous feedback is necessary for improvement in both 
teaching and learning. Teachers need to assess learning so that 
they may provide feedback to students on their progress as learn- 
ers* And teachers need to receive continuous and accurate feed- 
back on the impact of their teaching on the students in their 
classrooms, so that they may improve their teaching. 

3. Finally, if the improvement of learning is the goal, we 
need to know more about how students learn. Classroom teachers 
who know the structure of their discipline and who know — or 
should know — the problems encountered by students in learning 
it, are in the best position to contribute to knowledge on teach- 
ing effectiveness. Research on teaching and learning is moving 
in the direction of studying cognition and learning in the con- 
text of the subject or content taught, and we need the participa- 
tion of discipline-oriented faculty in assessment and research on 
teaching and learning so that we may know how to improve the pro- 
cess. 

The remainder of this paper is concerned with a review the 
research on f aedback in the classroom to determine its impact on 
the Improvement of teaching and i earring. My hypothesis is that 
the most effective form of assessment is one that is continuous, 
that occurs as close as possible to the scene of the action in 
teaching and learning (the classroom) , and that provides diagnos- 
tic feedback to both teachers and •students — to teachers on how 
they can improve their teaching; to students on how they can 
improve their learning. 



Framework for Feedback 

The possibilities for assessing performance and providing 
feedback in the classroom nay be illustrated by the two-by-two 
matrix shown in Figure 1. 
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FIGURE 1. 
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The most common feedback on academic performance is that from 
teachers to students (Quadrant 2). This includes grades at the 
end of the course, and feedback throughout the semester in the 
form of written and oral comments on student work. 



The second most common form of feedback is from students to 
teachers (Quadrant 3). Such feedback may be informal and subcon- 
scious on the part of the student, for instance, body language 
showing inattention, boredom, or confusion, or it may be formal 
and even standardized, as in student ratings of instruction. 
Increasingly, students' evaluation of teaching is a formal proce- 
dure. A recent survey by Peter Seldin (1984) showed that 67.5 
percent of more "than 600 liberal arts colleges surveyed used stu- 
dent evaluations in evaluating teaching performance — up from 
54.8 percent in 1978. 

Another form of student to teacher feedback that is most cen- 
tral to the current assessment movement is feedback on student 
learning outcomes. Learning outcomes for individual college 
courses are usually assessed by final exams or term papers at the 
end of the course. Although such feedback comes too late to do 
much about improving learning for that term, it may help the 
teacher in subsequent classes. 

A less formal, but equally important form of feedback from 
students to teachers comes in the form of student responses to 
assignments and questions posed in class. We don't often think 
of such information as "assessment." All too frequently, teachers 
assign work or ask questions, not with the intention of using the 
feedback to assess and diagnose, but rather to determine whether 
students have "done the assignment." Assessment via daily and 
weekly assignments is easily available to every classroom teacher 
at low cost, but it is undervalued and underutilized in the cur- 
rent assessment movement. Moreover, few teachers know how to 
design assignments that reveal the outcomes of student learning 
while the class is in progress. 
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Unfortunately, perhaps, most assessment of students by teach- 
ers and of teachers by students (Quadrants 2 and 3) is currently 
done in order to make judgments — what grade to give students, 
which in turn is used to make decisions about the student's aca- 
demic future, and what rating to give teachers, which is used to 
sake decisions about promotion and tenure. In either case, 
assessment becomes adversarial rather than developmental; it is 
sore likely to be used for accountability than for improvement. 

The remaining two forms of feedback illustrated in Figure 1. 
are teacher to teacher (Quadrant 1) and student to student (Qua- 
drant 4). Some teachers invite colleagues to observe and provide 
feedback on their teaching; the colleague may be a trusted friend 
or an "assigned" faculty evaluator. Also included in this cate- 
gory are consultants from offices charged with the improvement of 
instruction and certain technological forms of feedback such as a 
video tape in which a teacher provides feedback tc himself or 
herself. Finally, student to student feedback is used increas- 
ingly in collaborative learning, in student critiques of writing, 
etc. 

Ideally, the feedback in all four cells would be two-way com- 
nunication. The assessment model currently in use that comes 
closes* o this ideal is probably the program developed by 
Alverno College in which assessment is considered an inevitable 
part of the learning process. Individual student learning is the 
focus of interest, and teachers are constantly looking for indi- 
cations of progress on the part of learners (Loacker and others, 
1986, p. 52). 

The major emphasis in this paper will be on the cells that 
concern feedback £fi teachers from students and colleagues, that 
is, Column 1, which includes quadrants 1 and 3. This does not 
suggest that a paper emphasizing Column 2 would not be useful. It 
certainly would, bat this paper is a review of research on using 
feedback to improve teaching in the conviction that feedback to 
teachers on students' learning may be the most efficient and 
effective route to the improvement of learning. Moreover, there 
is a need in the current assessment movement for some alterna- 
tives to large-scale standardized testing. 



FEEDBACK FROM STUDENTS TO TEACHERS. 

There are two broad categories of feedback provided by stu- 
dents to teachers: 1) Student evaluations of teaching in which 
students report their observations and reactions to the course 
and instruction, and 2) Measures of student learning, such as 
tests, daily assignments, etc. Of the two, researchers have 
given more attention to the study of student evaluations of 
teaching thar. to classroom measures of learning. Much of this 
research has been concerned with the credibility of students as 
judges of teaching performance. Credibility becomes important 
when student evaluations are used as one bit of information use 



9 

o 

ERIC 



6 



ful in making promotion and tenure daciaiona. It ia alao eaaen- 
tial if taachara ara to banafit from commenta about how atudenta 
raapond to taaching. The former use ia in the accountabil- 
ity/ summative mode of assessment, whereaa the latter ia in the 
formative/ improvement mode. 

Major studies have been conducted over the paat decade of the 
potential aourcea of biaa and the reliability and validity of 
atudenta as evaluatora. By thia time, these queationa have been 
extensively atudied, with the generally-accepted concluaion that, 
while deciaions regarding promotion and tenure ahould not be made 
on the basis of student evaluationa alone, atudenta are, as a 
group, reaponsible and reliable witnesses to the quality of 
instruction offered ir» their classrooms. They can provide infor- 
mation that ia useful in improving the quality of undergraduate 
education, which is the end goal of assessment. Since the results 
of this research are a neceaaary, albeit not sufficient, condi- 
tion for determining the usefulness of student feedback to taach- 
ara, a brief review of the findinga will be provided. 

Three questions dominate the research on student ratings of 
instruction: 1. Are there aourcea of biaa in student ratings? 2. 
Are atudent ratings related to atudent learning, i.e. do atudents 
rate most highly those teachers from whom they learn the moat? 3. 
Are atudents consistent in their ratinga, i.e. are responses 
reliable and stable over time? At stake in all three queationa 
ia the place of student-evaluation information in a program of 
assessment . 

Following a synthesis of the research findings with respect 
to each of these questions, I shall offer some general obaerva- 
tions and recommendations for making better use of students' 
evaluationa of teaching as elements in an effective assessment 
program (see Page 16) . 



Sources of Bias In student Rating s of Instruction. 

Almost everything that anyone can reasonably propose as a 
. biaaing condition in atudent ratinga of teachera has been atu- 
died. Marsh (1980) , for example, looked at 16 variablea includ- 
ing expected grade, reason for taking the course, class level, 
course work load and teacher rank, concluding that all 16 vari- 
ablea accounted for only 12 tc 14 percent of the variance in stu- 
dent ratings. The major sources of biaa can be discussed under 
three categories: student characteristics, faculty characteris- 
tics, and course characteristics. 

1. Student Characteristics: student characteristics auch as 
gender, grade point average, college year, academic ability, rea- 
aon for taking the courae, peraonality, and age do not, on the 
whole, produce significant differencea in atudent ratinga of 
inatruction. (Centra, 1977; Coatin, Greenough, and Mengea, 1971; 
Centra and Creech, 1976). Centra (1977, p. 97) concludea that "In 
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general, the research indicates that most extraneous variables 
have a relatively weak relationship to ratings, ." 

Of major concern in rating programs used for decision-making 
is the question of whether students 9 expected grades in a course 
lead to lack of objectivity. Prey (1976) found no difference in 
ratings collected before and after the final examination. Ove- 
rall, the evidence seems to indicate that students do not "penal- 
ize" a teacher whom they think will give them a low grade — 
although some studies have shown that students may give lower 
ratings when they expect a grade lower than usual for them (Cen- 
tra and Creech, 1976) • 

For our purposes here it is responsible to conclude that 
there are not consistent or significant biases related to student 
characteristics. If bias does appear in an individual classroom, 
it should be thoughtfully considered by the teacher. For example, 
if low-ability students are more critical than high ability stu- 
dents, one possible explanation is that the teacher is giving 
more attention to high ability students. Similarly, since there 
are not significant differences in the ratings men and women give 
their teachers (Centra, 1979) , teachers finding a gender bias in 
their classrooms may first look to themselves as the source of 
the bias. 

2. Faculty Characteristics : Over the years, th»ie have been 
scattered but consistent complaints from faculty that "wit" and 
"personality" lead to higher-than-deserved ratings. There was a 
flurry of excitement over the "Dr. Fox" studies of the early 
1970 9 s which seemed to show that Dr. Fox — an actcr posing as an 
instructor with wit and personality but little content — 
received higher ratings for effectiveness than did the instructor 
who presented solid content. Subsequent studies have been unable 
to duplicate those results, but there is a consistent relation- 
ship between student racings and instructors and "enthusiasm" 
(Murray, 1985). 

The enthusiasm of the instructor almost always shows up in 
lists of characteristics of effective teachers (Abrami, 1985; 
Feldman, 1976) , but there should be no particular surprise in 
that. Getting and maintaining the interest and attention of stu- 
dents undoubtedly plays a role in motivation and achievement; as 
such enthusiasm probably should count as a relevant factor in 
teaching effectiveness (Murray, 1985) • Coats and Smidchens 
(1966) found that lecture material presented in a dynamic fash- 
ion, including gesture, vocal inflection, and minimal use of 
notes, was recalled significantly better than the same material 
presented with less enthusiasm. 

Feldman (1976) reviewed nearly 60 studies of students 9 
descriptions of effective teachers and found enthusiasm consis- 
tently listed among faculty characteristics that received high 
ratings from students. Other characteristics rated high in most 
of the studies were concern for students, knowledge of subject 
matter, stimulation of interest, availability, encouragement of 
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discussion, ability to axplain clearly, and preparation. It is 
encouraging that these characteristics, perceived by students as 
important to teaching, are behavioral and subject to change by 
the teacher. They are, in the terminology of Bloom (1980), 
"alterable variables," as opposed to "unalterable" characteris- 
tics such as race, gender, and age. 

Other faculty characteristics — less subject to change by 
individual teachers — that have been studied are years of expe- 
rience, academic rank, gender, teaching load, and research pro- 
ductivity. None of these characteristics shows consistent or sig- 
nificant relationships to student ratings, with the exception of 
those related to teaching experience (Centra, 1979). Teachers 
with three to twttlve years of teaching experience are rated some- 
what higher than those with either less or more teaching experi- 
ence (centra and Creech, 1976). while it is understandable that 
teachers with little teaching experience might have much to 
learn, there are implications for faculty development in the 
findings that teachers with 12 to 20 years of experience may need 
attention. Boredom, "burn-out," and obsolescence have all been 
discussed as deterrents to good teaching among experienced teach- 
ers. 

Ironically, teachers with heavy teaching loads — more than a 
13 credit-hour load — were given the highest ratings in one com- 
prehensive study (Centra and Creech, 1976). Generally speaking, 
these teachers are located at community colleges and four-year 
colleges with a clear teaching mission. While this may suggest 
that an emphasis on teaching produces results, it also raises a 
question about the point at which hours in the classroom begin to 
produce diminishing returns. I was unable tr> locate any research 
studies on this topic, which is of major concern to teaching 
institutions. 

Classroom behaviors, in contrast to faculty characteristics, 
flfi show significant correlations with student ratings. Behaviors 
that have been studied are information-giving, stressing impor- 
tant points, giving multiple examples, signaling the transition 
to a new point, and behaviors aimed at establishing rapport or 
encouraging student participation, such as asking questions, 
addressing students by name, and showing concern for student 
progress (Murray, 1985) . Such behaviors have the dual advantages 
of being alterable and observable, if observation can inform a 
teacher of weaknesses, then it may be possible to alter the 
classroom behaviors responsible. This is a finding that has con- 
siderable promise for assessment for improvement, when feedback 
about observable behaviors goes directly to teachers, they may 
then be able to improve their teaching — at least as it is per- 
ceived by students. 

3. Course Characteristics; The major course characteristics 
that have been studied for possible bias are class size and 
whether the course is required or elective. There is evidence of 
a small but consistent relationship between course size and stu- 
dent ratings — with small classes (fewer than 12 students, 
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according to Centra, 1977) recaiving eoaewhat highar ratinga. But 
aa Claaaon (1986, p. 12) notej, N Tha Kay ia tha modesty of tha 
ralationahip which, dapanding on tha atudy, accounta for anywhere 
from ona to aight parcant of tha varianca in ratinga — defini- 
tely not enough to uae large couraaa aa an excuae for low rat- 
inga." 

Student! alao tend to give aonawhat highar ratinga to courses 
in their majors — even if required — and elect ivea. The rala- 
tionahip, however, ia once again very modest. Gleaeon (1986, p. 
11) concludea that, "Required couraaa tend to be rated a little 
(aa in vary li-tle) lower than electives." 

In comparing thousands of classes across virtually all fields 
of atudy, Centra and Creech (1976) found that student ratings of 
course value and teacher effectiveness were slightly higher in 
the humanities than in the aocial and natural sciences. It may 
be aignificant that teachers in the humanities are usually some- 
what more likely than teachara in other fields to rate teaching 
aa more important to them than research (Parsons and Piatt, 
1968) . 

A review of the research on potential biasing factors of stu- 
dent evaluations of teaching should convince most faculty members 
that students can serve as objective and helpful sources of 
information about their teaching. Evan tha sir ill biases that do 
ahow up in the reaearch e.g. class size and role in the college 
curriculum, are related more to global ratings and comparisons 
with other teachers than to behaviors that teachers can change in 
their own classrooms. 

Before drawing general conclusions about the role of student 
evaluationa o. teaching in assessment, it may be helpful to 
review other reaearch on tha usefulness students as sources of 
information about teaching effectiveness. 



Consistency and Agreement on Teaching Effectiveness 

The aecond diaenaion of student evaluationa of teaching that 
haa been investigated extensively is the consistency or reliabil- 
ity with which atudent.s evaluate teaching. Centra (1979, p. 26) 
after studying atudent evaluations extensively, concludes that, 
"Their reliability or consistency, as indicated by numerous 
atudiea, ia very good, providing enough students in \ class have 
made the ratinga." 

Teachara, especially those who are not especially popular 
with atudenta, aomatimea complain that when students get more 
perapactive and maturity, they will come to value the teaching 
they received at the hands of a "tough taskmaster." The research 
offers little support for this rationalization, however. Cantra 
(1977, p. 101) concludes that tha research on this question 
clearly indicatea that "current students and alumni agree sub- 
stantially on effective and ineffective teachers." Correlations 
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of student ratings with alumni ratings hovsr around .75. It seams 
to make littla diffaranca in ovarall judgments vhathar ratings 
ara oollaotad during the ssmestsr, at the conclusion of the 
course, five years later or even ten years later, since alumni 
ratings are difficult to get and ara of dubious value in provid- 
ing feedback on current and apecific behaviors that have been 
found moat useful to instructors seeking improvement, there would 
seem to be little point in collecting alumni rec Elections on the 
performance of individual teachers. This is not to say alumni ■ 
follow-up can't be a valuable aourca of information about curri- 
culum, for inatance to provide information about the outcomes 
from general education and the major. 

Another queation that arisss frequently with respect to the 
value of student perceptions of teaching effectiveness is the 
question of whether students know good teaching when they see it. 
Do they know, for example, whether the instructor is current in 
the field, and whether they are learning what they need to know 
about the aubject? The answer to that queation ia that there are 
people better qualified than students to judge the adequacy of 
the syllabus, but the research shows that students, faculty, and 
administrators tend to agree on the identification of good teach- 
ers (Blackburn and Clark, 1975; Doyle and Crichton, 1978; Marsh, 
at. al. 1979; Centra, 1979). Blackburn and Clark (1975) found 
correlations in the low 60a between faculty, students, and admin- 
istrators in ratings of instruction. Correlations in other 
studies ranged generally upward. Naslow and Zimmerman (1956) 
found a correlation of .69 between colleagues and student ratings 
within a department, and Murray (1972) reported a correlation of 
.87 between colleagues and student ratings of 32 teachers. More 
will be said about the value of feedback from colleagues later, 
but for now, the point ia that the identification of good teach- 
ere ia not as difficult or as controversial as has sometimes been 
assumed. 

Validity 

Validity is the most important question of all. studies of 
validity address the question, Do students rate most highly these 
from whom they learn the most? Most studies do report signifi- 
cant relationships between ratings of teaching effectiveness and 
student learning. Correlations usually run between .40 and .60, 
although Dowell and Neal (1982) found correlations ranging from 
.87 to one negative corralation of .75. Cohen (1981), in a meta- 
analysis of 41 validity studies of course ratings with student 
achievement, reported a correlation of .43. 

The problem for those investigating the relationship between 
students* lsarning and their rating of instruction has been the 
determination of criteria for how much students learn in the 
courae — a central issus raised by assessment generally. The 
usual criteria uaed in research invaatigating the relationship 
between grades and student's ratings of teachers are scores on 
final exams (for multi-section courses) or some other form of 
common end-of -course examination. But other criteria such as 
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value of the course to students and continuing interest in the 
subject matter have also baan uaad. Cantra (1977) found tha 
highest corralations batwaan final exam grades and global ratings 
of "teacher effectiveness" and "value of tha course" to studants. 
Tha corralation batwaan tast scoras and ratings of tha valua of 
tha coursa to studants, for axampla, vara .73 in chamistry and 
.92 in biology. In contrast, corralations with studant percep- 
tions of coursa difficulty and workload showad tha lowest corre- 
lations with achievement. Apparently, student perceptions of the 
amount of work required bear little relationship to the amount 
learned. While students may take "gut" courses and courses where 
the workload is light and the grading is easy, there is no evi- 
dence that they rate them highly. 

Peter Cohen (1981) provides an appropriate summary for this 
section on what we now know from the voluminous literature on 
student ratings of instruction. He conducted a sophisticated 
seta-analysis of 41 studies that met his criteria for inclusion 
in a statistical synthesis. Re concluded that, "We can safely 
say that student ratings of instruction are a valid index of 
instructional effectiveness. Students do a pretty good job of 
distinguishing among teachers on the basis of how much they have 
learned" (p. 305) . While others are less optimistic about the 
validity of student ratings (Dove 11 and Neal, 1982), most of 
those advising caution do so, not on the grounds that teachers 
cannot get useful feedback from students, but that administrators 
should be cautious about depending on student evaluation in pro- 
notion and tenure decisions. Such advice argues for more atten- 
tion to the development of assessment for improvement models and 
suggests caution in the application of assessment for account- 
ability models. 

So far in this review of the research, one of the potentially 
strongest cells of our two-by-two matrix is the one discussed 
above, representing student feedback to teachers. Students are 
important sources of assessment and feedback about teaching 
improvement. They are present in the classroom throughout the 
semester, on good days and on bad, and no one knows better than 
students how the teaching affects their learning, motivation, and 
interest in the subject. If they were trained to be more astute 
observers of the impact of teaching on their learning, they might 
provo more helpful than they are now in providing feedback that 
is utieful in the improvement of both teaching and learning. The 
crux of the matter, however, lies in whether student observations 
do ox could improve teaching, i.e. do teachers change as a result 
of feedback from students? 



Do Student Ratings Improva Instruction? 

This question of whether feedback from student ratings 
improves instruction is, of course, the critical question for all 
assessors who claim that the purpose of assessment is to improve 
the quality of education. It should bs acknowledged immediately, 
however, that both rating forms and research have been designed 
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more for Baking judgments about teaching than for making improve- 
ments. Gleason (1986, p. 12) observes that while low ratings may 
motivate improvement, ratings that rank instructors against each 
other or measure them against a set of norms, are not specific or 
descriptive enough to tell faculty what aspects of instruction 
n*ed to be changed, a problem at the classroom level which manif- 
ests itself at the level of state-wide rankings of programs and 
institutions, as well. Hurray (1985, p. 22) illustrates the point 
when he writes, "As things stand now, an instructor who receives 
poor ratings on global dimensions such as clarity and rapport may 
have no idea of the specific behaviors that led to these ratings 
or of the specific changes that need to be made to bring about 
improvement." 

If student evaluations of teaching are to be useful in the 
assessment-for-improvement model, they will have to be essen- 
tially redesigned to provide feedback that is useful to teachers. 
This may mean a review of criteria suggested in the beginning of 
this paper, i.e. that feedback for improvement should emphasize 
competencies and should be non public and noncompetitive. While 
teachers might be more likely to change their behavior as a 
result of such feedback, there is still the question of whether 
they can and do change. We turn to that now. 

The research that exists on change as a result of feedback is 
mixed, but encouraging. Centra (1973a) found that when teachers 
thought they were doing a better job than their students thought 
they were doing, they did change, that is teachers who received 
what they interpreted as criticism in the form of low ratings did 
make changes, and they did so in as little as half a semester. 
Centra also found that a wider variety of instructors changed if 
given more than half a semester of time, and if they were given 
help in interpreting the feedback. The changes that were most 
likely to occur were in these areas: preparation for class, use 
of class time, summarization of major points in lectures or dis- 
cussion, openness to other viewpoints, and making helpful com- 
ments on papers or exams. These are all specific behaviors, sub- 
ject to change by the instructor. 

Murray (1985) reports on two studies that attempted to 
improve teaching through feedback providing information about 
specific classroom behaviors. Like Centra (1973b), he reported 
that teachers who needed to change (i.e. who were ranked low by 
students) showed more improvement than those who ranked higher, 
and they showed improvement in as brief a period as six weeks. 

There are several explanations for these findings. One is 
found in dissonance theory (Festinger, 1957) , which maintains 
that change is most likely when the instructor perceives a large 
discrepancy between self-ratings and ratings by students. The 
r * su ,^ according to this theory, is an attempt to restore 
equilibrium by moving toward closuro of self -perception and the 
perceptions of others. Sullivan (1983) proposes a different 
explanation, observing that progressing from poor to adequate 
teaching usually requires the elimination of specific behaviora" 
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weaknesses. This suggests that it is probably sasier to move a 
tsachsr from poor to adequate performance than from adequate to 
outstanding teaching. And indeed, that has been the conclusion 
of most consultants on teaching improvement. In any case, feed- 
back from students would appear helpful to teachers who are not 
doing well. 

The research to date suggests that teachers, in particular 
teachers who are rated low by students, can improve as a result 
of feedback alone. Much remains to be done, however, in improving 
feedback procedures if assessment-for-improvement is the goal. 
Gleason-Weimer's observations (1987, p. 9), unfortunately, still 
apply to all too many colleges using student evaluations. She 
writes: 

"Evaluation results (if faculty get them back) are 
returned via some ... impersonal, albeit efficient 
method. Generally, results come back to faculty via 
the mail.... They come with varying amounts of stat- 
istical cybernetics to decipher and varying degrees 
of helpful instructions.... One we know lists all 
sixty faculty members by the last four digits of 
their social security numbers and then rank orders 
them from top to bottom by their overall rating of 
effectiveness. To b- last on such a list is devas- 
tating. Being tenth from the bottom is hardly encour- 
aging. And to what end? The comparison may indeed 
motivate faculty, but if the data do not help them 
identify specific areas in need of alteration, and 
if no opportunities to discuss the results are provid- 
ed, faculty may be motivated to become defensive, not 
better teachers." 

Unfortunately, Gleason's descriptions of the uses of feedback 
for the improvement of teaching apply all too frequently to the 
uses of feedback in assessment programs generally. When data 
collection and quantification become the goals, feedback serves 
no useful purpose and may even damage the motivation to improve. 

The findings here suggest that when the quality of feedback 
is given attention, implementation is a likely result. Cohen 
(1980) conducted the seminal analysis on what we know about the 
impact of student feedback on teaching performance. He located 22 
research studies comparing student ratings of instruction under 
conditions of feedback and no feedback. In a meta-analysis, syn- 
thesizing the results of these studies, Cohen concluded that the 
feedback group received higher end-of-term global ratings in 20 
of the 22 comparisons. The improvement was sufficient to place 
the typical instructor in the feedback groups at the 65th percen- 
tile, compared with the 50th percentile of the instructors in the 
control groups. But even more interesting is his finding that 
"augmented** feedback, which included consultation about the rat- 
ings, resulted in an average increase which placed the teacher at 
the 74th percentile (in the judgement of students) by the end of 
the semester. If colleges could raise the performance of teach- 
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ers from the 50th to the 74th percentile in a semester by using 
student evaluations augmented by consultation, the result would 
be a really massive improvement in the quality of instruction, 
and by inference, in students' learning. This constitutes the 
strongest possible argument for continuing to work toward an 
assessment- for-improvement model, through the uses of feedback in 
the classroom. 



Feedback tQ Teachers on Student Achiev ement In vh« classroom. 

I had hoped in this paper to review the research on whether 
instruction improves as a result of Knowledge about student 
achievement. The assessment movement, after all, is based 
largely on the premise that if college faculties know what stu- 
dents are and are not learning, they will take steps to improve 
the situation. There is, however, very little research on which 
to Judge the impact of feedback about student learning outcomes 
on the improvement of instruction. 

In his search of the literature, Cohen (1980) found only 
three studies that used student achievement measures in feedback 
experiments. He also located very few studies using student out- 
come measures such as student perceptions of their learning and 
student attitudes toward the subject area. In all cases, his 
meta-analysis suggested that the comparisons favored the feedback 
groups, but the results were not generally statistically signifi- 
cant. I suspect that the less than significant results are 
related more to our failure to design strong and useful models of 
assessment-for-improvement than to the failure of people to 
respond to concrete suggestions for improvement. I think it is 
safe to assume that most college teachers want to do the best 
possible job of teaching. To date, we (researchers, assessors, 
administrators, and policy makers) have not been very helpful in 
providing the type of information and feedback on performance 
that permits teachers to engage in useful self -evaluation. Much 
work remains to be done in designing better assessment and feed- 
back and in investigating the question of whether and how teach- 
ers change their teaching to accomplish greater learning. 

For the past tw* years, I have been speaking and writing 
a»out the potential of Classroom Research to inform teachers what 
students are learning in their classrooms (Cross, 1986; 1987). 
The potential of Classroom Research to improve practice, however, 
rests on the assumption that we can devise feedback devices that 
will provide information to teachers about student learning and 
that teachers can act on that information to improve learning. 

Some of the classroom feedback devices that Tom Angelo and I 
have described our new handbook on classroom assessment tech- 
niques (Cross and Angelo, 1988) solicit the type of student reac- 
tions to teaching and course materials that have been discussed 
above, but we are also trying to develop classroom assessment 
techniques that inform teachers about student achievements in 
subject matter learning, critical thinking and analysis, and cre- 
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ative thinking and synthesis (Section I of the Handbook) • The 
classroom assessment techniques in Section I, for example, are 
designed to provide continuous feedback to the teacher during the 
semester (formative evaluation) on how veil students are learning 
to analyse or to "break down" a large topic, question, or problem 
in order to understand it more fully and manipulate it more 
effectively. Also included in Section I are suggestions for 
assessment techniques that provide information about students' 
abilities to create original intellectual products and to syn- 
thesize concepts and ideas from different sources into a coherent 
whole. The emphasis in the techniques described in the Handbook 
is on making assessment a part of the everyday activities of the 
classroom, using assessment in homework assignments, and incorpo- 
rating assessment devices into teaching methods. 

We also think it is important for classroom teachers to get 
feedback on how students think about themselves as learners and 
how they assess their own learning skills (Section II of the 
Handbook) . The classroom assessment techniques in Section II 
address the recent highlighting of the need for students to 
become involved in the active process of learning. "Metacogni- 
tion N is the term used to describe students 1 understanding of 
their own learning skills, performance, and habits. If teachers 
had better feedback from students about how students think about 
themselves as learners, teachers might modify their teaching. 
Understanding how students engage themselves in the process of 
monitoring their own learning, for example, should be helpful to 
teachers in making assignments and designing class experiences to 
encourage active student involvement. It is one thing to recom- 
mend that students be actively involved in learning; it is quite 
another to suggest to teachers how that can be accomplished. 

Unfortunately, most teachers are more concerned about what 
they, the teachers, are doing than about what students are doing 
in the classroom. It would be both enlightening and frightening 
to know just how active student minds are during the typical lec- 
ture. Section III of the Handbook which includes assessment 
techniques designed to provide feedback on student reactions to 
course materials and teaching methods, is more akin to some of 
the topics addressed above under student evaluation of teaching. 

Our efforts to develop feedback devices that inform classroom 
teachers about the reactions of students to their teaching — in 
terms of achievement, attitudes, and teaching evaluations — are 
in the nascent stage of development, but research to date cer- 
tainly suggests that teachers can improve in response to feedback 
from students. Certainly, it seems worth the effort to begin the 
development of feedback devices that will make a difference. 
Ultimately, I think faculty members will have to form their own 
networks and to become involved in developing the sort of feed- 
back on students 1 learning that is most useful in their particu- 
lar disciplines or to accomplish their particular teaching goals. 
In the meantime, however, those of us interested in assessment- 
for-improvement can attempt to illustrate how feedback might be 
made more useful and more effective. 



ERLC 



13 



-16- 



Conclusions and Recommendations fo r the Improvement of 
Instruction Throug h the Use of Feedback From Students. 

1. In general, students provide relatively unbiased, 
reliable, and valid Information about the effectiveness of teach- 
ing. Although much of the research has been designed to answer 
questions about the usefulness of student evaluations for deci- 
sion-making purposes, the findings are sufficiently positive to 
suggest that if improvement of teaching is the goal, students can 
serve as one important source of constructive feedback to teach- 
ers in the classroom. Two conditions are helpful in gaining use- 
ful feedback: 1) Faculty must consider student evaluations cre- 
dible and useful, and 2) The information would be even more use- 
ful if students were taught to be careful observers of the impact 
of teaching on their learning. 

2. If student feedback is to be maximally useful, student 
rating forms should be redesigned to stress specific behaviors 
that are subject to change by the teacher. There should also be 
provision for teachers to select or add items that are of partic- 
ular interest to them, since there is no need for forms used for 
self -improvement to be standard, teachers should be helped and 
encouraged to develop their own methods and devices for obtaining 
feedback. 

3. Feedback from students appears most effective for those 
who need it most — those who think they are doing a better job 
of teaching than students think they are doing, since the discre- 
pancy between self -ratings and student ratings seems to be a 
motivating factor in change, It may be desirable to encourage 
self -ratings and explicit comparison with student ratings of 
instruction. 

4. Feedback is significantly more effective in producing 
change when it is augmented with consultation. Trained consul- 
tants should be provided to faculty and their use encouraged. 

5. Research is needed on the efficacy of providing teachers 
with better measures of classroom learning. In particular, 
attention needs to be given to the development of better indica- 
tors of classroom learning and to whether and how teachers use 
feedback from studant achievement measures in the improvement of 
instruction. 

6. campuses committed to using assessment for improvement 
should seriously consider a range of methods that utilize feed- 
back from students to faculty. 
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COLLEAGUES AS SOURCES OF FEEDBACK 



The notion of a community of colleagues working together to 
improve education has strong roots in academe, and many college 
teachers say they are more influenced by colleagues than by stu- 
dents. Braskamp (1985) found in interviews with 60 faculty mem- 
bers at a large university that "feedback from others was not 
only used, but sought after." (p. 548). This was especially true 
for assistant professors where feedback from faculty colleagues 
carried special weight in the formative years. 

Feedback on teaching, however, is a special situation — 
seemingly more sensitive and perhaps less useful than feedback on 
research or other professional activities. There is some ques- 
tion about how colleagues can realistically provide feedback 
about teaching. In my study conducted at the University of 
Nebraska in the mid-1970 's, I found that teachers were generally 
reluctant to have colleagues visit their classes. Only 28% were 
very willing to have colleagues visit their classes, and 42% were 
hesitant or would object (Cross, 1977) . Peer evaluation of class 
materials, however, is quite acceptable; 47 percent of the 
Nebraska faculty endorsed it and another 35 percent were "willing 
to consider it." 

There is also the practical question raised by Centra (1979) 
about how much time colleagues can spend observing classes. Two 
visits a term by one or two colleagues would seem the maximum 
that evaluators would be willing or able to give — unless the 
evaluator was acting as a friend, consultant, or special adviser. 
This is the special circumstance under which feedback from col- 
leagues as peers will be discussed here. 

Finally, there is the question of whether colleagues possess 
the necessary skills in observation and consulting to be helpful. 
Centra (1979, p. 84) observes that, "Little improvement comes 
from occasional class visits by colleagues or administrators who 
do not know what to look for or who many not be particularly 
effective teachers themselves.... On the other hand, observa- 
tions by skilled experienced colleagues or teaching/course 
improvement specialists can be extremely useful." 

Research and experience with colleagues as sources of feed- 
back on teaching can be discussed under three headings: 1) Col- 
leagues as Peers, 2) Colleagues as Consultants/Experts, and 3) 
Feedback via Technology. As in the previous section on student 
feedback to teachers, colleague feedback can be used in both 
assessment- for-accountability and assessment-for-improvement mod- 
els. Ironically, it seems that the most rudimentary and superfi- 
cial colleague assessments (e.g. a single class visit by the 
dear) are often used for accountability, whereas the more inten- 
sive analyses (e.g. team teaching or close friends as consul- 
tants) are used in assessment-for-improvement models. 
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Feedback from Colleagues as Paers. 

The uaa of collaaguaa as paara in providing faadback dapanda 
haavily on truat. It would appaar that aoat auch collagial 
afforta ara informal, voluntary, spontaneous, and fraquantly gen- 
aratad among paopla who ara alraady frianda — all of which laads 
to thair lack of credibility in aasessment-for-accountability, 
but which may have high potential for use in assessment-for- 
improvement. 

Team teaching in which colleagues work together on planning, 
teaching, and evaluation is almost always a rich learning experi- 
ence, and some groups of faculty have extended this by meeting in 
small groups to explore teaching issues, consider each other's 
teaching methods, and to discuss teaching methods (Hoyt and How- 
ard, 1978) . This approach can be used with or without classroom 
visitations. Sweeny and Grasha (1978) formalized this procedure 
by forming "peer development triads" in which teams of three fac- 
ulty members work together to help each other assess and improve 
their teaching. 

Katz and Henry (Katz, 1987) have developed an intensive and 
long-term approach to faculty development that combines colleague 
and student observations with some of the other forms of feedback 
that were described earlier, in their model, teacher, colleague 
observer, and students are all working constantly at atimulating 
greater awareness and knowledge of the teaching/learning process, 
especially as it is related to the personal development of stu- 
dents. Katz reported that faculty in some 20 institutions where 
the method has been tried found "a gain in energy and pleasure in 
teaching, in part because they acquire a surer knowledge of their 
impact on students and a new colleagueship with other faculty 
members and with their students" (p. 30) . 

Other forms of generating feedback from colleaguea as peers 
have required administrative support, often in the form of 
arranging opportunities, but sometimes requiring financial 
resources and released time as well. 

Peter Elbow (1980) describes a Danforth-supported project in 
which one faculty member was freed each quarter to spend time 
visiting faculty members who had volunteered to be visited 
— which, according to Elbow, consisted of more volunteers than 
could be accommodated. Elbow's sensitive description of his 
experience contains a detailed analysis of how he as visitor 
approached some sensitive issues in providing feedback on the 
teaching of a colleague. He concluded that, "What this approach 
requires is participanta who care about teaching and about each 
other and who are willing to look closely at what they see and to 
report accurately how they respond. The process is built on 
trust, but our experience leads us to believe that trust flows 
naturally from the structure of the procedure — as long as safe- 
guards are observed" (p. 40). * 
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Other sodela of faculty development that dapand on paar feed- 
back caat teachers in the rolea of atudenta — sometimes for one 
course, aoaetimea for an entire semester as a atudent taking a 
full load of coursea. Roy Starling (1987) deacribea with wit and 
insight the semester he spent as a student (maeter-learner) in 
Rollins College Community of Learners program. Hia commentary 
conaists of observations not ao much on the delivery of classroom 
teaching aa on its perception by students. He wrote, "On certain 
daye, I couldn't wait to return to teaching to put my new-found 
knowledge to work; on other day a, the prospect of trying to fix 
so such that was broken led me to consider less demanding profes- 
sions." (1987, p. 3). In this particular instance, Starling's 
feedback took the form of a group of resolutions that he made to 
himself about how to improve hia own teaching. 

Some programs in which teachers serve as students also empha- 
size the importance of feedback from teacher/ students to the 
teachere in whose classes they enroll. The experiment devised by 
Sheila Tobiaa in cooperation with the Univeraity of Chicago 
(Tobias, 1986) illustrates this perspective. She set forth the 
interesting notion that non-acience faculty members could provide 
useful information to science teachers about the difficulties 
experienced in learning a new subject, especially in science and 
mathematics. Professors from other disciplines, she reasoned, 
would have few of the problems sometimes attributed to regular 
studenta having problems with subject matter — youth, lack of 
confidence, inability to concentrate, etc. The only characteris- 
tic these sophisticated learners would have in common with under- 
graduate atudenta would be naivete about the subject matter. 

When two phyaica professors at the University of Chicago gave 
demonstrations and lectures to their non-scientist peers, much 
was learned by both sets of professors. Two levels of feedback 
were provided — one during and immediately following the class 
sessions, and a aecond in a reflective letter composed anywhere 
from two daya to two months after the experiment. The observa- 
tions of the non-scientist "students" ranged from insights into 
their own behavior as leamera to newly discovered relationships 
to their owri fielda of expertise, to constructive suggestions to 
the lecturers. One observed, for example, "I waa surprised by my 
reactions to these lectures in the following ways: first, by how 
very interesting I found what I was able to understand and, aec- 
ond, by how quickly when I failed to understand something immedi- 
stftly >y usual feelings of mind-lock, frustration, panic, and 
helplessness surfaced" (Tobias, 1986, p. 38). Another 
teacher/student found that he lacked the disciplinary perspective 
to help hia distinguish basic concepts from what was being commu- 
nicated merely for the purpose of illustration. Zn thia instance, 
feedback to the inatructor about the importance of providing a 
map of the terrain seemed important. 

There are few formal atudiea of whether feedback from col- 
leagues as peera results in changes in teaching, informal testi- 
monials are often quite impressive — if the relationship and 
trust between peera really worked. Many campuses report that the 
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most valuable consequence of assessment thua far ia increased 
quantity and quality of faculty interaction, cooperation, and 
diacouraa about teaching. Moat professional working across a 
broad range of people and colleges are convinced that feedback 
from colleaguea can be enhanced by a few relatively simple proce- 
dures. 

Some officea of inatructional improvement have prepared out- 
linea to guide the obaervation of colleaguea and make them more 
uaeful. (See Centra, 1979, pp. 85-89 and Clift and Imrie, 1981, 
pp. 122-124 for some examplea.) othera conduct special training 
aeaaions to help guide the observationa and consultations 
(Graeha, 1977). As uaeful as theae may be, there ia some evi- 
dence that the uae of experienced and professional consultants is 
probably more efficient and effective than the training of non- 
apecialiat colleaguea. Centra (1979, p. 84) found that "formal 
or informal colleague assessments are less effective than consul- 
tation with expert faculty or work with master teachers." We turn 
now to the uae of feedback from trained consultants. 

2. Feedback from Consultants and Experts. 

The 1970s waa a decade of growth nationwide for offices of 
instructional improvement. Centra (1976) reported that 41 per- 
cent of the four-year colleges responding to his mid-1970s survey 
claimad "an on-campua person or unit(s) for faculty development 
or inatructional improvement...." (p. 34). since many people 
wondered if the fiscal austerity of the 1980s had taken its toll 
among these college-supported services, Glenn Erickson (1986) 
conducted a similar aurvey in 1985. He concluded that, "We have 
no evidence here to auggest that faculty development is dying" 
(p. 183). Forty- four percent of the respondents in four-year 
institutions reported offices of instructional improvement, and 
66 percent claimed that their institutions' current investment in 
faculty development waa much or somewhat greater than it had been 
three years earlier. Among the most frequent services available 
to faculty under the category of "individual help from a trained 
consultant" were help with use of instructional technology (66%) , 
interpreting student ratings of instruction (46% — although 95% 
of all respondents reported student ratings of instruction were 
in uae on their campuses), developing teaching skills, e.g. ask- 
ing questions, lecturing (45%), course planning or development 
(43%) , and constructing tests or evaluating student performance 
(43%). 

There ia broad agreement in the literature that expert con- 
sultation is an important element in feedback to teachers (Cen- 
tra, 1979; McKeachie, at. al, 1980; Menges and Brinko, 1986; 
Habel, et. al. 1988), and almost equal agreement on the nature of 
aome basic guidelines, in a nutshell, the recommendations are 
that the relationship with a consultant muat be voluntary, the 
feedback confidential, and the focus on description and improve- 
ment rather than on evaluation and remediation. All of these 
recommendationa carry the familiar earmarks of the criteria for 
feedback in the assessment- for- improvement model. 
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It also appears that intensive feedback, coordinatad and com- 
ing from a variaty of aourcaa is mora af factiva than any aingla 
affort. Skaff (1983) found that physic ian/taachara who racaivad 
intanaiva feedback (videotaping, plus studant and staff ratings, 
plus a salf-assassmant quastionnaira, plus a ona-hour confaranca 
with a taaching consultant) vara far mora likaly to rata thair 
experience "def initaly banaf icial" (75%) than vara taachars vho 
racaivad only a singla form of faadback (6% to 12% ratad it dafi- 
nitaly banaf icial) . Moraovar, thosa racaiving intanaiva faadback 
vara significantly mora likaly to idantify and improva taaching 
bahaviors. Thasa findings auggast that assessment- for- 
improvamant is going to ba a mora intanaiva and long-tarm invast- 
mant than a ona-shot collaction of data, follovad by committaas 
to maka racommandations for improvement. Thay also suggast that 
coordinatad af forts, orchastratad by an offica chargad vith the 
rasponsibility for improving instruction ara likaly to ba mora 
af factiva than sporadic af forts by volunteers. 

Most consultants have learned through experience hov to give 
feedback that is acceptable and uaeful to teachers. There is rel- 
atively little research in education on hov to present feedback, 
although some conclusions from business varrant pondering. Tosti 
(1978) questions the common visdom of giving feedback immediately 
and coupling praise vith criticism. He distinguishes betveen 
"motivational 19 feedback and " format ive" feedback. Motivational 
feedback, ha contends, should be positive and immediate, vhereas 
formative feedback should be given, not immediately folloving 
performance but immediately prior to an opportunity to practice 
nev bahaviors. 

If Tosti 1 s observations concerning relatively simple tasks in 
industry (telephone performance of salesmen) apply to the more 
complex behaviors of teacher* in the classroom, it vould appear 
that the best time to conduct formative teaching assessments is 
during the semester. Most feedback, especially student ratings 
and student learning outcomes, are available to teachers only at 
the end of the semester, frequently vith at least a summer or 
semester intervening before the course is taught again. One of 
the advantages of consultations vith experts and peers may be 
that these typically come during the semester vhen the instructor 
can try out nev approaches. Since research suggests that evalua- 
tions of teaching are generally quite stable after the first fev 
veeks of class, there is little reason for vaiting until the end 
of the semester to gather reactions to teaching (Costin, 1968; 
Kohlan, 1973). 

The Office of Instructional and Management Services at the 
University of Illinois (1987) recommends that faculty conduct 
their ovn "early semester feedback. " They suggest four vays of 
obtaining such feedback: an informal questionnaire (3-12 scaled 
items plus 1-3 open-ended questions) ; written comment (3-4 open- 
ended essay-type questions) ; a group interviev (an educational 
specialist interviews all or a sample of students in the class 
during 10-15 minutes of class time; and a quest ion/ansver session 
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(oral questions possd to students during the first or last 10 
minutes of class.) 

One of the most interesting and creative projects for provid- 
ing feedback from an office of instructional improvement is one 
designed by Robert C. Wilson of the University of California at 
Berkeley (1986) . He combined almost all the services that offices 
of instructional improvement typically perform into an innovative 
package designated, " Persona A Teaching Improvement" (TIPS). 

Starting with student ratings of instruction, the office pre- 
pares a printout for each instructor of student ratings on each 
of 24 items in the student rating scale. Special attention is 
called to the two items rated most descriptive and the four items 
rated least descriptive of the class. 

If the instructor wishes to change low-rated items, he or she 
is directed to the Teaching Improvement Packet, which is enclosed 
with the teaching evaluations. This packet consists of sets of 
suggestions indexed to items on the student rating scale. For 
example, a teacher who receives low student ratings on an item 
reading "knowing if the class is understanding you or not" is 
directed to a four-page flyer offering ten suggestions for what 
teachers might do to assess the level of students' understanding. 
Typical of the suggestions are these examples: 

1. Assign "minute papers" at the end of some lectures. Min- 
ute papers ask students to write their answers to two questions: 
(1) What is the most significant thing you learned today? and (2) 
What questions remain uppermost in your mind at the end of 
today's class session? 

2. Call on your students to paraphrase or summarize what you 
have just said. 

3. Encourage your students to form small study groups arl 
send representatives to see you about the difficulties the groups 
are having. 

„°I} 0 of m ? 8t interesting features of Wilson's approach is 
that the suggestions included in the TIPs flyers come from Berke- 
ley faculty members rated especially high on the characteristic. 
Staff from the teaching improvement office interviewed faculty 
who received especially high ratings on the student rating 
scales, asking, "Can you think of anything you do that would lead 
students to say that it is very descriptive of your teaching that 
you (the item is inserted here, e.g. explain clearly; know if the 
class is understanding; encourage class discussion, etc...)." The 
suggestions, coming from their colleagues in the classrooms at 
Berkeley, are likely to have high credibility, practical utility, 
and demonstrated effectiveness. This is an especially intriguing 
model in which expert consultants combine feedback from students 
and feedback from faculty colleagues to offer practical sugges- 
tions on areas that the teacher needs and wants to work on. 
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An emerging role for off ic** of insl ructional improvement is 
that of consultant on Classroom Research. The concapt of encour- 
aging classroom teachers to collect information about student 
progress in learning has garnered an impressive array of descrip- 
tive nomenclatures recently — classroom research (Cross and 
Angelo, 1988) , practice-centered inquiry (Chism and others, 
19*8) , course-embedded assessment (Farmer, 1988) , and reflection- 
in-pr&ctice (SchHn, 1987), among others. 

I observe the emergence of three roles for professional con- 
sultants in these classroom-based inquiries, each with its own 
strengths and weaknesses. The first ir to provide research ser- 
vices for teachers. In this model, teachers are encouraged to 
propose experiments and researchable questions that professional 
researchers can then investigate. Harvard and Syracuse, among 
others, are currently conducting research illustrating this 
model. The Harvard Assessment Semirar, for example, is presently 
conducting research on the effectiveness of different approaches 
to teaching a ccrse in moral development. Since this is a core 
course which enrolls some 400 students, research on the most 
effective ways to teach it are likely to have substantial impact. 
Bob Diamond, long-time director of the Center for Instructional 
Improvement at Syracuse University, offers as part of the ser- 
vices of his office, the design and conduct of classroom research 
proposed by Syracuse faculty. 

The advantages and disadvantages of Model 1 are fairly clear. 
Because the research is conducted by specialists in educational 
research, it is likely to be competent and credible. The problem 
is that there is a heavy demand on resources which is on-going 
because faculty are not being trained to do their own classroom- 
based inquiry. Moreover, faculty are not necessarily becoming 
more astute and knowledgeable observers of student learning. They 
are interested in the results of the research, presumably, but 
they are not gaining the insights and experience that comes from 
doing their own classroom research. 

Model 2 is a consultative model on classroom research in 
which teachers are encouraged to develop their own research pro- 
jects with the help and assistance of experts. This model is 
currently in use at The Ohio State University and the University 
of Rhode Island. (See Chism and others, 1987 and Erickson and 
Erickson, 1988) . The consultative model has the obvious advantage 
of giving faculty "ownership 19 in the research, and it also scores 
an advantage in gradually training faculty to do their own 
research, as well as to become more knowledgeable about matters 
pertaining to teaching and learning. Experience to date, however, 
indicates that, at least in the beginning, many faculty propose 
highly complex and difficult questions that are likely to call 
for traditional social science research designs which the faculty 
member is not really qualified to carry out. Discouragement and 
reinvention of the wheel are possible results. 

The success of Model 2 would appear to depend on reaching 
some happy balance between expert direction on how to "do it 
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right" and permissiveness to "create your own thing." The 
reality is that tha mora diractiva tha conaultant, tha mora it 
becomes tha conaultant 'a pro j act and responsibility, incorporat- 
ing moat of tha advantagaa and diaadvantagaa of Modal 1. A per- 
miaaiva, non-diractiva atanca, howavar, runs tha risk of dis- 
couragement and disillusionment for faculty who lack tha tine, 
background, and axpartiaa to carry out tha complex projects that 
thay mora of tan than not daaign. 

I hava baan experimenting with the development of a third 
model that I call Classroom Research. It consists of providing 
some relatively simple tools that classroom teachers, without 
training in social science research, can use to get started on 
the systematic observation of student learning in their class- 
rooms. The Handbook of Classroom Assessment Techniques referred 
to earlier (Cross and Angelo, 1988) is a compendium of 30 teach- 
ing/assessment techniques that can be used by teachers in any 
discipline to get systematic feedback on the impact of their 
teaching on students' learning. He are also experimenting with a 
Teaching Goals Inventory (TGI) that helps teachers make their 
instructional goals explicit and hopefully directs their atten- 
tion to what to look for in student responses to their teaching 
(Cross and Fideler, 1988) . 

He believe that the chief advantage of Model 3 is the gradual 
training of faculty members to become skillful and resourceful 
observers of student reactions to teaching as it takes place in 
their particular discipline. The major disadvantage to date is 
that sophisticated researchers — and therefore many faculty mem- 
bers — are likely to discount the simplicity of the techniques 
roposed and to regard the proposals for Classroom Research as 
ust one more fad in higher education. Our strategy and our hope 
is that faculty members will get started by using some of the 
simple strategies that we offer in the Handbook as examples of 
what can be done easily and inexpensively in the regular course 
of teaching. After this general introduction to classroom assess- 
ment, we hope that teachers will gradually develop more sophisti- 
cated assessment measures appropriate to their own disciplines. 

This will entail the development of discipline-based networks 
wherein teachers may exchange ideas about the assessment of stu- 
dent learning in their particular disciplines. The spinoff from 
this type of assessment may have implications far beyond the col- 
lection of information. Meetings of high school, community col- 
lege, and university teachers of English in a geographical region 
may have implications for articulation as well as assessment. 
Workshops for teaching assistants may encourage the development 
of classroom assessment techniques that are adopted or modified 
by professors. Departmental faculties may meet to discuss class- 
room teaching goals made explicit by some instrument such as the 
Teaching Goals inventory (TGI) and then go on to develop appro- 
priate techniques to assesa the accomplishment of such goals. At 
the moment, there is ample room for all three models, and expert 
consultants need not make choices; they can use the methods that 
best meet their needs and resources. 
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Zn conclusion, both research and experiance indicate that 
teachers can derive conaidarabla banaf it from working with col- 
leagues on taaching i»»ua». All indications ara that tha poten- 
tially most profitabla partnerships ara trained professionals 
from offices for the improvement of instruction and trusted 
friends or mentors who are willing to give time and attention to 
the task. Training faculty members to be more systematic and 
helpful observers of teaching probably comes in a distant third, 
and assigning faculty members or administrators to observe class- 
rooms without any special training for the task is probably a 
dubious procedure that has little to recommend it. 

3 . Sal f -Asaassmant . 

Final! v e should look at the potential of self-assessment 
to provide _*eful feedback to teachers in their classrooms. The 
very scanty literature on self -assessment, compared with the 
voluminous literature on assessment by others, gives testimony to 
the dominance of interest in assessment for accountability or 
decision-making over assessment for improvement. Self -assessment 
is suspect in promotion and tenure decisions because it carries 
the implication of bias or lack of objectivity — a conclusion, 
by the way, that gets some support from the research. Centra 
(1973b) found that 30 percent of the teachers in a five-college 
study rated themselves higher than students rated them. And I 
found in my study of the University of Nebraska faculty that 94 
percent of the respondents to a questionnaire about teaching 
rated themselves "above average" as teachers, and 64 percent 
judged themselves to be in the top quarter with respect to teach- 
ing performance (Cross, 1977) • 

Despite these findings suggesting that teachers tend to give 
themselves favorable ratings, there is also some evidence that 
people do have useful insights into their own performance. 
Although the teachers in Centra's study used primarily the posi- 
tive end of the rating scale, they did do quite well in identify- 
ing their relative strengths and weaknesses as observed by stu- 
dents. The rank order correlation between the ordering of items 
by self and by students was .77, indicating that teachers were 
able to see in themselves, albeit through rose-colored glasses, 
roughly the same strengths and weaknesses that students saw. 

Some offices of instructional improvement have encouraged 
self -assessment and have developed check lists and self -appraisal 
forms that help teachers look for behaviors in themselves that 
are known to affect teaching. Items on the list may be as simple 
and objective as "Do I explain how final grades are determined?" 
to as complex and subjective as "Do I try to design the course to 
meet the needs of students of varying backgrounds and abilities?" 
When Erickson (1986) conducted his study of the faculty develop- 
ment practices in some 650 four-year colleges, he found that 
about a third of the respondents reported systematic self- 
assessment procedures in use on their campuses. That seems sur- 
prisingly high to me — until we compare it with the use of stu- 
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dent ratings of instruction (96 psrcsnt) and classroom observa- 
tion by pssrs (64 psrcsnt) . 

Vary rscsntly, there has bssn a resurgence of interest in 
sslf -appraisal. Zt shows up in thrss distinctivs threads in the 
current literature on education: metacognition, reflection-in- 
action, and the uses of technology to enable people to observe 
themselves in action. 

The concept of metacognition involves self-observation and 
sslf -monitoring of the learning process by the learner. Good 
learners tend to be conscious of themselves as learners and to be 
able to observe, direct, and control the learning process. Poor 
lsarnsrs, in contrast, ars less obssrvant or analytical about the 
stsps or procedures that thsy follow in lsaming (Weinstein and 
Undsrwood, 1986; McKsachis and Others, 1986). Since teaching is a 
lsaming experience, we might reason that a teacher who is able 
to analyse the process of learning the complex skills and 
insights that go into tsaching would bs in a batter position to 
dirsct and control the process. What we are learning about meta- 
cognition should apply to learning to teach as well as to any 
other learning task. 

A rslatsd phenomenon is advocated by Donald Schon (1983; 
1986) . He believes that the best way to educate people for a 
variety of professions ranging from architecture to psychoanaly- 
sis is through rsflsction-in-action, which he defines as people 
"thinking what they are doing while they are doing it" (Schon, 
1986, p.xi). 

The concept of the "reflective practitioner," which has gen- 
erated considerable interest in professional circles, has much in 
common with ths concspt of metacognition, which has generated a 
great deal of excitement among educational researchers. Both 
rsquirs lsarnsrs to observe themselves in action, to become aware 
of how they process information and learn new skills. Neither 
Schon nor those inter* id in mstacognition assume, however, that 
the^learner can perfc these analyses without sxpsrt help. 
Schon, for example, devotes a grsat dsal of attention to the need 
for "coaching," and researchers speak of ths need for "modeling" 
and "talking through" the processes of observing oneself in 
action, whether in ths process of learning or in the process of 
psrforming profsssional tasks. At this point, the notion of self- 
improvement by sslf-monitoring looks promising. Certainly those 
designing assessment programs should not rule out the potential 
of delivering valuable feedback from oneself to oneself. 

While there is not much research that can be reported as yet 
on the more immediate and analytical forms of sslf observation, 
such as metacognition and r elect ion- in-act ion, there is a rela- 
tively long experience with using ths tschnology of audio and 
vidso recording which permits teachers to observe themselves, if 
not during the action, then in a faithful reproduction of it. 
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Videotaping of classes has bean one of the services offered 
by college and university offices for instructional improvement 
for years. Video playback made its appearance in the mid-1950s 
when it received its most enthusiastic reception from therapists, 
athletes, and entertainers. Today, the use of video playback is 
somewhat controversial in therapy (Fuller and Manning, 1973) , but 
thoroughly familiar to TV spectators who watch intently the slow 
motion video playback of the sacking of the quarterback. For 
tennis instructors and golf pros, video playback is the backbone 
of their instruction. 

The use of video has been less spectacular in higher educa- 
tion, but it is an inevitable part of microteaching, which is a 
basic pre-service training technique for elementary/secondary 
teachers. Centra (1976a) found that by 1976, 57 percent of the 
colleges and universities in his sample had video equipment 
available, but fewer than five percent of the faculty actually 
used it. Brickson (1986) reported that about half of the four* 
year colleges used videotaping with critique as a practice in 
faculty development programs. I don't think these data mean that 
videotaping is declining in use, but there has been no dramatic 
spurt in its use, and it has lagged behind such practices as stu- 
dent evaluation of teaching, which jumped from 55 percent used by 
four-year liberal arts colleges in 1978 to 68 percent by 1983 
(Seldin, 1984). 

The use of video is controversial, but much of the research 
and much of the controversy arises around its use in therapy, 
where it is usually discussed as " self -confront at ion. " There is a 
noticeable absence of research on the effectiveness of video 
playback in faculty development programs. From the research in 
psychology and what little has been done in higher education, we 
can draw the following tentative conclusions: 

1. Video playback, like any assessment-for-improvement, is 
probably most useful when the focus is on very specific, alter- 
able behaviors. The more carefully defined the goal, the more 
likely the change. 

2. Since self confrontation is a demanding emotional experi- 
ence, video playback should be accompanied by experienced consul- 
tation. 

3. It appears that there are arguments for combining video 
and audio playbacks. Video is often a dramatic, motivating expe- 
rience, but change seems not to persist over time. Fuller and 
Manning (1973, p. 492) write that, "listening to one's voice is 
like being given a little shake, but self -viewing is like a tee- 
thrattling blow. 11 They suggest that once people have seen a 
video, they can visualise what is going on from the audio alone. 
Since audio is easier and cheaper, it can provide continuous 
feedback that seems to be as effective as the more dramatic video 
feedback. 
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4. There is some evidence that video feedback is most effec- 
tive with secure, self-confident people. Fuller and Manning 
(1973) coapare the teacher most likely to benefit from video 
playback with the client most likely to benefit from therapy. The 
most promising client, they say, is the "YAVTS: young, attractive 
(and perhaps anxious, or at least in some pain), verbal, intelli- 
gent, and successful. If the YAVIS is a ferala, so much the bet- 
ter. The client least likely to benefit is the HOUND: homely, 
old, unattractive, nonverbal, and dumb" (p. 489). 

While it might be assumed that college faculty are generally 
secure, self-confident people, teaching seems to be a sensitive 
and personal matter. While faculty are eager to circulate their 
latest publication, or even first drafts, to colleagues across 
the nation, 42 percent of the Nebraska teaching faculty, which I 
judged quite typical in most respects, would be "quite hesitant" 
or would "object" to having colleagues visit their classes 
(Cross, 1977). I might observe that most people feel insecure 
about a public display of skills for which they have been given 
no training — even if they have watched others teach all their 
lives. 

5. since attention in the first sessions of video playback 
is usually given to an intense focus on self — voice, appear- 
ance, and mannerisms — the more productive playbacks are likely 
to occur in second and third sessions where attention can be 
directed to teaching behaviors. 

Given the potential of using feedback from oneself to improve 
performance, tna idea of self -assessment through the use of tech- 
nology is appealing, but it probably needs more research and more 
thoughtful analysis of what it can and cannot do and how its use 
can be maximally beneficial. 

Conclusions 

The advocates of assessment hope it will make a difference. 
The essential assumption behind that hope is that if we have an 
accurate assessment of students' learning, we can act to improve 
the quality of education. It is not an unreasonable assumption, 
but it is an unexamined one. 

This paper is, in one sense, a mini-evaluation of the assump- 
tion underlying assessment. This review concerns small-scale 
assessments, specifically the assessment of teaching in the 
classroom. Over the years, we have amassed considerable research 
about the response of teachers to feedback about their teaching. 
We have some information about who can provide useful feedback, 
what kind of feedback seems most helpful, and whether educators 
(in this case, teachers) change as a result of feedback. All of 
these issues are relevant to the larger-tcale assessments that 
are so pop-alar today. Although I hope this review of the 
research will be helpful in designing and improving large-scale 
assessment programs, that is not why I wrote this paper. 
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I wrote it because of ay growing conviction that assessment 
is taking place too far from the scene of the action. Once we 
get the information from institutional and statewide assessments, 
we are going to have to figure out how to use it to improve edu- 
cation. The consistent experience in education is that a huge 
gap exists between research and its eventual use to improve prac- 
tice. The leap from "research" to "development" is often so 
great that people cannot make it easily. We may find a similar 
chasm between assessment and the improvement of education. 

An alternative strategy to current models of assessment is to 
start on the side of the chasm where we would like to see change, 
thus making the leap between assessment and practice less formid- 
able. Since faculty members are the designated guardians of 
standards and quality, it is they who should be doing the assess- 
ments, and it is they who will have to make the changes in curri- 
culum and inrtruction. 

My particular attention in this paper has been directed to 
the uses of assessment to improve instruction. Changing instruc- 
tion is usually considered more difficult and intractable than 
changing the curriculum. While changes in the curriculum are 
revisited with the regularity of seven-year locusts, instruc- 
tional improvement has received only sporadic attention. Yet what 
happens when teachers meet students in the classroom lies at the 
heart of educational quality. 

One of the most encouraging findings that turned up in this 
review was reported by Cohen (1980) • In a meta-analysis of 22 
research studies investigating the power of feedback from student 
ratings to improve instruction, he found that relatively small 
improvements made by individual instructors added up to very sub- 
stantial improvement. He concluded that the typical instructor, 
utilizing feedback from students (in consultation with a consul- 
tant on teaching improvement) , raised his or her ratings from the 
50th to the 75th percentile by the end of the semester. It would 
be hard to think of any policy resulting from large-scale assess- 
ments that would bring about that amount of change is so brief a 
period of time. Granted, the improvement was in students 9 per- 
ceptions of instruction, but my review of the extensive amount of 
research that has been done on the reliability and validity of 
student ratings is quite positive. Students are one good source 
of information about the quality of instruction that is delivered 
in their classrooms. 

Although the current assessment movement rests on the assump- 
tion that learning outcomes are the truly valid indicators of the 
quality of education, relatively little research has been done on 
whether teachers change as a result of receiving information 
about student performance on tests and other measures of student 
achievement. Further investigation of this question is essential 
for the assessment movement generally, and it is crucial for the 
concept of classroom research. 
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My assumption in proposing classroom research is that if 
teachers engage in continuous assessment of what students are 
learning in their classrooms, teachers can then experiment and 
develop more effective teaching strategies. That remains to be 
seen. But a first step is to help classroom teachers conduct 
assessments of students' learning in their own disciplines, with 
the type of students they teach, and on matters that are relevant 
to their teaching goals. Then we need to find out if and how 
teachers respond to such assessment information. 

In conclusion, the purpose of any assessment is to improve 
the, quality of education. My argument in the review and inter- 
pretations presented in this paper is not necessarily to replace 
the current emphases on large-scale assessments with classroom 
assessments, but to offer a promising alternative to the efforts 
already underway. 
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ADDITIONAL RESOURCES 

AAHE ASSESSMENT FORUM 



The following resources are available for purchase from the AAHE Assessment Forum: 



-"Assessment, Accountability, and Improvement: Managing the 
Contradiction," P. Ewell 

••"Assessment and Outcomes Measurement: A View from the 
States," C. Boyer, P. Ewell, J. Finney, and J. Mingle 
••'The External Examiner Approach to Assessment," B. Fong 
••"Six Stories: Implementing Successful Assessment," P. 
Hutchings 

••"Thinking About Assessment: Perspectives for Presidents and 
Chief Academic Officers," E. El-Khawas and J. Rossmann 

2. Resource Packet IL Six Papers $25.00 

••"Acting Out State-Mandated Assessment: Evidence from Five 
States," C. Boyer and P. Ewell 

••"Assessing Student Learning in Light of How Students Learn." J. 
Novak and D. Ridley 

••"Faculty Voices on Assessment: Expanding the Conversation," 
P. Hutchings and E. Reuben 

-"Feedback in the Classroom: Making Assessment Matter," K. 
Patricia Cross 

-"Standardized Tests and the Purposes of Assessment," J. 
Heffernan 

-"An Update on Assessment," (AAHE Bulletin, December, 1987), P. 
Hutchings and T. Marchess 

3. Three Presentations- 1987: $8.00 
from the 2nd National Conference on Assessment in Higher Education 

-Lee S. Shulman - "Assessing Content and Process: 
Challenges for the New Assessments" 
-Virginia B. Smith - "In the Eye of the Beholder: 
Perspectives on Quality" 

-Donald M. Stewart - "The Ethics of Assessment" 

4 lliree Presentations-lS 38: $10.00 
from the 3rd National Conference on Assessment in Higher Education 
-Alexander Astiu - "Assessment and Human Values: 
Confessions of a Reformed Number Cruncher" 
-Linda Darling-Hammond - "Assessment and Incentives: 
The Medium ic the Message" 
-Robert H. McCabe - "The Assessment Movement: 
What Next? Who Cares?" 

6. Assessment Programs and Projects: A Directory $10.00 
Concise descriptions of thirty assessment projects being 
implemented on campuses across the country. Edited by Jacqueline 
Paskow. 

To order items indicated above, contact: Patricia Hutchings, Director, AAHE Assessment 
Forum, One Dupont Circle, Suite 600, Washington, DC 20036; 202/293-6440. Orders under 
$26 must be prepaid. Allow four weeks for delivery. MAKE CHECKS PAYABLE TO AAHE 
ASSESSMENT FORUM. 



L Resource Packet L Five Papers 



$15.00 
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